In a load-balanced configuration, a load-balancer system receives requests for service from numerous client systems. Depending on the load-balancing algorithm used therein, the load-balancer sends a received request to one of the systems participating in the load-balanced group of systems serviced by the load-balancer. The system that receives the request from the load-balancer processes the request, or queues the request for processing, depending on the conditions and configuration of the system.
A system that is participating in the load-balanced group can be a virtual machine (VM). Any reference to a system is a reference to a VM unless expressly distinguished where used. A VM includes virtualized representations of real hardware, software, and firmware components available in a data processing system. The data processing system can have any number of VMs configured thereon, each VM utilizing any number of virtualized components therein. The data processing system is also referred to as a host, and the VM operating thereon to process a client request is also referred to as a computing node, a compute node, or simply a node.
A commonly used method of virtualization—traditional virtualization—preconfigures various VM configurations as template images (templates). When a VM having a specific predetermined configuration has to be created on a compute node, a suitable template is selected from a template storage, such as a database or a file-system, and installed on the compute node to create a VM having the desired configuration. An image of a VM in traditional virtualization is a monolithic binary image.
Another method for virtualization is container-based virtualization. Container-based virtualization, also called operating system virtualization, is an approach to virtualization in which the virtualization layer runs as an application within the operating system. In this approach, the operating system's kernel runs on the hardware node with several isolated application environments installed on top of it. The isolated guest application environments are called containers. In other words, a container is a running instance of an image of a VM in which the layers are distinguishable from one another.
Hereinafter, a virtualized computing entity (VCE) is a reference to a VM or a container, depending upon the implementation. The illustrative embodiments are described with respect to VMs only for the clarity of the description and not to imply any limitation thereto. The illustrative embodiments can be adapted for applying to a load-balanced group of containers, and such adaptations are contemplated within the scope of the illustrative embodiments.
A virtual switch or vSwitch is a virtual networking component that operates in a manner similar to a physical networking switch device. For example, a vSwitch determines a packet's destination and provides the destination address and port-based packet forwarding function to the interfaces utilizing the vSwitch. Unlike a physical switch, which is a hardware device, a vSwitch is an instantiation of a virtual object from the execution of software designed to operate in the manner of a physical switch.
As an example, one or more VMs executing on a host data processing system can be configured to interface with a vSwitch executing on the host. The vSwitch receives data packets from the interfacing VMs and sends the packets to the datacenter network with which the vSwitch also interfaces. Similarly, the vSwitch receives data packets from the datacenter network—e.g. from a load-balancer—and sends the packets to the appropriate interfacing VM destination.
Presently, if a system in a load-balanced group has excessive utilization of the processor, the memory, the network bandwidth, the electrical power, or some other computing resource, the system can offload the request to another peer system in the group. Such presently available offloading is dependent only upon the utilization-related conditions of one or more computing resources in system.
Some other presently available offloading algorithms look at the number of requests pending in a queue in the system. When that queue-size exceeds a threshold, the presently available offloading algorithm offloads one or more requests from the queue to shorten the queue. A request is interchangeably referred to herein as a “job” unless expressly distinguished where used.